Alphabet: Loop Tower

Loop Calculus introduced in [1], [2] constitutes a new theoretical tool that explicitly expresses the symbol Maximum-A-Posteriori (MAP) solution of a general statistical inference problem via a solution of the Belief Propagation (BP) equations. This finding brought a new significance to the BP concept, which in the past was thought of as just a loop-free approximation. In this paper we continue a discussion of the Loop Calculus. We introduce an invariant formulation which allows to generalize the Loop Calculus approach to a q-are alphabet. The manuscript is organized as follows. In Section I we introduce a new formulation of the Loop Calculus in terms of a set of gauge transformations that keep the partition function of the problem invariant. The full expression contains two terms referred to as the “ground state” and “excited states” contributions. The BP equations are interpreted as a special (BP) gauge fixing condition that emerges as a special orthogonality constraint between the ground and the excited states. Stated differently, it selects the generalized loop contributions as the only ones that survive among the excited states. In Section II we demonstrate how the invariant interpretation of the Loop Calculus, introduced in the Section I, allows a natural extension to the case of a general q-ary alphabet. This is achieved via a loop tower sequential construction. The ground level in the tower is exactly equivalent to assigning one color (out of q available) to the “ground state” and considering all “excited” states to be colored in the remaining (q− 1) colors, according to the loop calculus rule. Sequentially, the second level in the tower corresponds to selecting a loop from the previous step, colored in (q−1) colors, and repeating the same ground vs excited states partitioning procedure into one and the remaining (q − 2) colors, respectively. The construction proceeds until the complete set of (q − 1) levels in the loop tower (including the corresponding contributions to the partition function) is established. In Section III we discuss an ultimate relation between the loop calculus and the Bethe free energy variational approach of [3]. We start with defining a statistical inference problem using the so-called Forney-style graphical model formulation [4], [5]. The basic graph, C0 = (V0, E0), is described in terms of vertices, V0 = {a} and edges, E0 = {(ab)}. Variables, associated with the edges, assume their values in a q-ary alphabet, σab = σba = 0, · · · , (q − 1). The probability of a given configuration of variables σ = {σab|(ab) ∈ E0} on the entire graph is described by p(σ) = Z C0 ∏